Skip to content

feat: derive execution timing from a summary lifecycle plan - #482

Open
zzylol wants to merge 3 commits into
feat/lifecycle-candidatesfrom
feat/lifecycle-timing
Open

zzylol wants to merge 3 commits into
feat/lifecycle-candidatesfrom
feat/lifecycle-timing

Conversation

@zzylol

@zzylol zzylol commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #476.

Rebuilt into the linear stack on main. Conflict resolutions are recorded in the messages of: "test: carry a chosen lifecycle through timing to physical compilation".

Why

Agreed layering: the summary maintenance lifecycle layer sits between logical
Post-ASAP and physical compilation, and a chosen lifecycle assignment is what
decides each node's execution timing (precompute vs query time). Today timing is
hard-coded by logical strategies when they build candidates, and a lifecycle
choice (e.g. Ephemeral vs ContinuouslyMaintained) never reaches the DAG, so
it cannot change what runs in precompute.

What

  • SummaryMaintenanceLifecyclePlan::execution_timed_dag() -> Result<PostAsapDag, SummaryMaintenanceTimingError>
    applies the plan's lifecycle choices to every node via with_execution_phases:
    • a retained state (ContinuouslyMaintained, Shared, Prepared) and every
      node feeding it run at ingestion time;
    • readouts, other consumers and Ephemeral states run at query time; the
      deployment supplies their raw data as a query-time source;
    • an Ephemeral state feeding a retained state runs at ingestion time (query
      time may not feed ingestion time);
    • a state without a selected lifecycle, or a maintained population the plan
      does not list, is refused.
  • Design doc: the "placement ownership is an open question" paragraph is replaced
    by the four-layer contract (logical → lifecycle → physical → deployment).

Hard-coded timing in logical strategies is left in place for a follow-up; with
Planner's default selection the timed DAG equals today's for the PromQL fixtures.

Before this PR

ContinuouslyMaintained or Ephemeral  →  same DAG, same precompute contents

After this PR

let plan = candidates.select(&[(state, SummaryMaintenanceLifecycle::Ephemeral)])?;
let timed = plan.execution_timed_dag()?;   // Ephemeral: everything at query time
// physical compilation: precompute empty, raw source read at query time

Validation

Unit tests for every lifecycle kind, a shared state with two readouts, an
Ephemeral state feeding a retained state, unselected and unplanned states.
End to end (summary_maintenance_lifecycle_e2e): Planner's selection reproduces
today's timing for quantile_over_time, quantile and sum by(job)(rate); a
Continuous choice puts exactly the state in precompute while Ephemeral leaves
precompute empty, and both return the same p99. cargo fmt --check and
cargo clippy --workspace --all-targets -- -D warnings pass; asap-aware-mapping
478 passed; workspace passes except the pre-existing
devtools dag_export::workload_node_ids_make_smaller_topk_reuse_explicit.

Follow-ups

🤖 Generated with Claude Code

zzylol and others added 3 commits September 30, 2026 18:05
A chosen summary-maintenance lifecycle did not reach the DAG: timing was
fixed by realization strategies. SummaryMaintenanceLifecyclePlan::
execution_timed_dag assigns every node's timing from the selection: retained
(non-Ephemeral) states and their inputs at ingestion time, everything else at
query time. States without a selected lifecycle, and maintained populations
that lifecycle enumeration does not cover, are refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rebase note: #483 removed the pane test this test was appended after; the
new test is appended to the #483 version of the file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the open question about placement ownership with the agreed four-layer
contract: logical Post-ASAP decides what to compute, the chosen lifecycle
decides timing, physical compilation partitions by timing, and the backend
prices lifecycle choices including store cost.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Sep 30, 2026
Replace the largest-non-ASAP-subtree fragment export with one Relational node
per operator, so physical compilation corresponds node by node. Name the
timing source as physical design (summary materialization) and point the open
LifecycleAssignment question at SummaryMaintenanceLifecyclePlan (#482).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the feat/lifecycle-candidates branch from 4bb8c86 to 07244ea Compare September 30, 2026 18:27
@zzylol
zzylol force-pushed the feat/lifecycle-timing branch from 0747971 to 3b281da Compare September 30, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant